4 Operating Procedures This section explains how to operate the four iBroker Interface components that are installed at the client site. 4.1 Extractor The program to run this is called M2AExtractor.exe and is located within the C:\Program Files\m2Controller\bin directory on the Systems Administrator Workstation. A shortcut called to this program called M2Extractor will be created for the Client's M2 Administrator by M2 operations staff on the Administrators PC Desktop. This program must be manually started by the Client every time an extract is to be run. It is the first and most critical aspect in the extraction of data from the in-house system and its transferral to M2. 4.1.1 Overview of Extractor When Extractor is started you will see the following. " Initially a screen will appear with the words M2A Extractor at the top. Beneath this will be 2 tabs labelled 'Operator' and 'Sys Admin'. The Operator tab will initially be enabled. At the top of this screen will be 4 drop down boxes labelled from top to bottom; Select Extract Type, Client(s), Data Range from and Date Range to. Beneath the drop down boxes is a display panel which displays information as the Extractor procedure runs. At the bottom are 2 buttons labelled Logout and Extract Now. " The second tab labelled Sys Admin can be activated by clicking on the Tab. Once again the screen has a section at the top which enables Extractor to be configured and a display panel beneath which displays messages. This tab is normally only used by M2 Support staff and rarely used once the system has been set up. The Sys Admin tab provides set-up and maintenance for the following items: " iBroker DSN - (The DSN is the Data Source that has been set up separately in the ODBC data sources section of the PC which you are using to point to the in-house Oracle Database storing the iBroker movements). The DSN will be set up for you by M2 or your Systems Administrator and should not be changed after initial setup. If problems occur, do not change this entry without first checking with your Systems Administrator or M2 staff. " User Name and Password - These are the User Name and Password required to provide access to the iBroker Oracle database. These will be created for you by the in-house iBroker system Database Administrator and should not need to be changed once initially configured. " Test DB Connection button - This enables you to ensure that the source (iBroker) and target (M2ARawInput) databases can be accessed and that the settings are correct. " iBroker Interest Code for M2 - This is a field used in iBroker to identify the clients to be 'synched' with M2. This will be set up for you by the in-house iBroker system Database Administrator and should not need to be changed once initially configured. " Save button - This enables all the above entries to be saved for future runs of the Extractor module. " Extract Step by Step - This is not currently available and is to be implemented in a future release. However, when it does become available there will be 2 options, Yes and No. The purpose of this field is to enable a step by step extraction of movements or an extraction of all movements in a single operation. Initially, a step by step extraction is advisable to ensure that the process is smooth. " Status Display window which is used for display the status of the Extraction process. " Last Movement Date:Time - The iBroker application date/time-stamps each movement as it occurs and this is used to keep track of which movements have already been extracted. After each daily Extract run, the most recent Extract Date:Time is stored to use as the start point for the next daily run. This field may be edited but this should not be done without first noting down the current value. You should also be very clear about what you are trying to achieve before changing this date. " Next - The Next button is not currently available and is greyed out. It will be used in a future release. The Operator Tab is used to run daily or ad hoc extractions of movements from the iBroker system. Whenever the Extractor runs, the Operator tab is activated and the program immediately tests the connections to both databases (as defined by the Sys Admin tab). This will alert the Operator should there be any basic problems, before the Extract is run. If the display panel indicates a problem, do not proceed until this is resolved. Currently there are two extract options: " Daily Movements - This will typically be run once per day in the morning (or if part of the EOD procedures after the iBroker EOD processing is complete) and extracts all movements from iBroker for M2-tagged clients (as per the iBroker Interest code for M2 setting) which are time-stamped more recently than the date stored from the previous run. In fact, this option can be run as often as you like as it may pick up additional movements during the day. However, these will be cash entries only as iBroker generates buy and sell movements as part of the EOD process. " Client History Load - This enables the initial load of historical movements for individual clients. This is typically used when a client is first tagged to be included within M2 (via the iBroker Interest Code for M2). A date range may be entered (If no dates are entered then All Movements will be extracted) for the movements that are to be loaded from iBroker. It is important that this is done by some-body who is actively 'managing' the take-on of existing portfolios as it is essential that the creation of 'opening positions' is fully understood and completed first as this may affect the selected starting date (See comments below on Business Procedures). When the 'Extract Now' button is pressed all of the extracted movements (in their original format) are displayed in the status display panel. They have now been loaded onto the RawInput queue awaiting the Adaptor process. At the end of the Extract process, information is displayed showing that the Extract Process is complete and the number of Records (movements) that have been processed (extracted). The configuration is then saved. The important part of this save process is that the Last Movement Date:Time from iBroker is recorded to be used later as the starting point for further extracts. 4.1.2 How to set up Extractor To set up the Extractor follow these steps. Once the Extractor is set up by M2 Support Staff, no changes should be required to these fields, and neither should they be undertaken without the direct involvement of M2 Support Staff. " Start Extractor from the Desktop icon on the PC and click on the Sys Admin tab. " iBroker DSN - Enter the iBroker DSN into this field, as supplied by the Clients Systems or Database Administrator. This will be a short name which matches exactly the name set up within the ODBC area on the PC on which you are running Extractor. The DSN provides the path to the iBroker database. " User Name and Password - Enter the User Name and Password into these fields. This information will be provided by the iBroker Database Administrator. " RawInput database DSN - No data is required to be entered in Extractor for this currently. However, the Extractor process looks for a database called M2ARawInput.gdb in the C:\Program Files\m2Controller\database directory on the local PC and this is used by M2 to store the extracted movements. For this reason the Extractor process should always be run on the Server PC. In future the movements will be stored within the m2asset database and will be accessed via the m2asset DSN. " Test DB Connection button - Click on this button after completing the above steps. This will confirm that the source and target databases can be accessed. The status will be displayed in the display panel at the bottom of the screen. If there are any problems displayed then these will need to be fixed before moving forward. " iBroker Interest Code for M2 - Enter the Interest Code that has been set up in iBroker into this field. This information will be provided by the iBroker Database Administrator or the person responsible for the configuration of iBroker. All clients tagged in iBroker with this code will be available for movement extract. " Save button - Click on this to save all of the above settings. 4.1.3 How to run Extractor for Daily Movement load To run the Extractor for a Daily Movements load (before starting, ensure that any new clients that have been tagged within iBroker for an initial load are run separately using the Client History Load option after following the business procedures for new client set up). " Start Extractor from the Desktop (the Operator tab will be displayed). Check the display panel to ensure that there are no errors with the database connections. The application will connect to the two databases using the DSN for iBroker. If the ODBCM2Adaptor connection is unsuccessful it will be because this database cannot be found within the C:\Program Files\m2Controller\Database directory on the PC on which you are running the Extract. In each case a '….. DB connection successful' message will be displayed if the connection is successful. " From the drop down box select 'Daily Movements', if it is not already displayed. " The Client(s) field will be greyed disabling the entry of any Client codes. The words 'All M2 Tagged Clients' will be displayed in the field. " The Date Range from field will be greyed. The word 'Latest' will be displayed in this field. The date that is used for the start of the extract is the Last Movement Date:Time which is displayed at the bottom of the Sys Admin tab. " The Date Range to field will be greyed. " Click on the Extract Now button at the bottom right of the screen. As the movements are loaded into the M2 RawInput database they are shown in the display panel. A message is also displayed giving the number of records that have been processed (extracted). If there has been any problem with the setup you have entered or with the connection, the Extract Now button will be greyed and you will not be able to run this step. " Logout of Extractor by clicking on the Logout button. 4.1.4 How to run Extractor for ad hoc (initial or one off) load To run the Extractor for an initial load of the end-user historical movements, follow these steps (ensure that the Business Procedures at the end of this document are followed first). " Start Extractor from the Desktop (the Operator tab will be displayed). Check the display panel to ensure that there are no errors with the database connections. The application will connect to the two databases using the DSN for iBroker. If the ODBCM2Adaptor connection is unsuccessful it will be because this database cannot be found within the C:\Program Files\m2Controller\Database directory on the PC on which you are running the Extract. In each case a '….. DB connection successful' message will be displayed if the connection is successful. " From the Select Extract Type drop down box select 'Client History Load'. " In the Client(s) drop down box, enter the Client Code which matches the IBroker Client Code. This will be supplied to you by the iBroker settlements clerk or Administrator (in the future you will be able to click on 'All M2 Tagged Clients' if you are ready to load movements for all the newly tagged clients; but for now ignore this option). Until this code is entered the Date Range to field will be greyed. When you enter the Client Code the Date Range from field is activated but before you can enter a date the code is validated against iBroker and the Account Name is displayed. If it is invalid a message saying 'Client does not exist in iBroker' will be displayed. " Ensure that the iBroker Interest Code for M2 on the Sys Admin Tab is correct and that the Client has been set up within iBroker for the extraction (Interest Code has been set). Ensure once again that the end-user starting positions have been set up in M2CRM. " If you have entered a specific iBroker Client Code successfully, select either "All Movements" from the drop down box or if you wish to run an ad hoc load for a specified period click on the From option to start the date picker which enables you to select a From date for the commencement of movements extract. Alternatively you are able to type in a date in an accepted date format. " If you have entered a Date in the Date Range from field then you will be required to enter a date in the Date Range to field. Use the date picker to select the final date for the extraction process. If you wish to extract all the movements up to today, click on today in the date picker. " Click on the Extract Now button at the bottom right of the screen. As the movements are loaded into the M2 RawInput database they are shown in the display panel. A message is also displayed giving the number of records that have been processed (extracted). If there has been any problem with the setup you have entered or with the connection, the Extract Now button will be greyed and you will not be able to run this step. " Logout of Extractor by clicking on the Logout button. 4.2 Adaptor In the future, the Adaptor process will be started as a Service under the control of the m2Controller application, which has been installed on the Server PC. However, for now responsibility for starting and running this Service will be with the in-house designated M2 system administrator. The program to run this is called M2Adaptor.exe and is located within the C:\Program Files\m2Controller\bin directory on the Server PC. A shortcut to this program called M2FAdaptor will be created for the M2 Administrator by M2 support staff on the System Administrators workstation. 4.2.1 Overview of Adaptor For now, while this step is manual, when the Adaptor is started you will see the following. Initially a screen will appear with the word FM_Main at the top. Beneath this will be 2 tabs labelled Messages and Submit. The Messages tab will be displayed initially. The Messages tab enables the user to display (and if necessary modify the data or the Status) the individual movements that have been extracted from the in-house system. " In the display panel at the top of the Messages screen will be displayed all (every message for every extraction, not just the latest extraction) of the movements that have been extracted from the in-house system using the Extractor application. Beneath this panel will be 4 drop down boxes which enable the user to select only those movements which match the selection criteria. " The first of these drop down boxes is labelled 'Created' and allows the user to show all movements (All Dates) or the movements for a selected date which can be entered directly or via the date picker. Initially this is set to 'All Dates'. " The second drop down box labelled 'Source' enables the user to display only those movements that came from a single source (for example iBroker) or All Sources. This selection would only be changed when the client had an Interface for a system other than the in-house iBroker system, for example a Cash Management system. Initially this will display 'All Sources' if there is more than 1 source or the source name if there is a single source only. " The third drop down box is labelled 'Filename' and allows the user to display all files (a file is created by each extraction) or just a single file (single extraction). The file names indicate the date and time the extract was run. The filename has a prefix (for instance BP or EX) and then the date (in yymmdd) format and then the time (in 24 hour format of hhmm). Initially this is set to 'All Files'. " The final drop down box is labelled 'Status' and enables the user to show only those movements with the matching status. The status may be 'Ready to Submit', 'Submitted OK', 'Rejected', 'Blocked' or 'ByPassed'. Initially this is set to 'All Statuses'. " To display a sub-set of movements only, select the options you want from the drop down boxes and then click on the 'Search' button. " At the bottom of the screen (in yellow) is displayed the currently selected movement (the movement highlighted at the top of the screen) status details and at the very bottom the actual data in the movement in the raw (original iBroker) format. To change the currently selected movement, use the cursor to click on a movement. You are able to scroll up and down using the scroll bars to the right of the display panel. " In the middle at the bottom is the 'Modify' button which enables the user to modify the data within an individual message. To do this (this should not normally need to be undertaken and should only be used by someone with intimate knowledge of the Interface process and requirements), click on the movement in the display panel at the top (white area) that you wish to modify. The detail for this movement is displayed in the yellow area. Then click on the 'Modify' button. This will change the yellow display panel at the bottom to white and you will be able to edit the data by clicking on the movement and using the keys on the keyboard to select the data in error and to fix it. Once you have completed the modification, click on the 'Save' button to save the edits or the 'Cancel' button to discard the changes. Only movements that have a status other than 'Submitted OK' can be modified. " Under normal circumstances you will only be interested in those new messages that have been extracted from iBroker and are ready to be submitted. To display only those messages that are ready to be submitted, click on the down arrow to the right of the 'Status' Drop down box and then click on 'READY TO SUBMIT'. Then click on the Search button and only those movements that are ready to be submitted will be displayed. Once you are satisfied with the movements and you have selected those movements that are Ready to Submit, click on the Submit tab at the top of the screen to undertake this process. The 'Submit' tab enables the user to submit the messages to the XML queue ready for transfer to the M2 service. This process involves writing the movements in an XML format into a queue. " When you go into this tab only those movements that are 'READY TO SUBMIT' will be shown in the white display panel at the top. " On the Submit tab there is a drop down box labelled 'Source' and three radio buttons. The three radio buttons are mutually exclusive (only one of them can be selected at any one time). The 'Source' is the database from where the movements were extracted (iBroker) and in nearly all cases this will never change. In fact, if there is only 1 source possible then this field will display the source and be greyed out meaning that it cannot be changed. " The radio buttons enable you to specify how to process the movements. You can process all 'Ready to Submit' messages in a single process, process the 'Ready to Submit' messages one at a time or process a selected message only. For now it is best (but slow) to click on 'one by one'. 4.2.2 How to set up Adaptor There is no requirement to set up the Adaptor application except that it requires Java JRE to be installed. Specifically it requires jvm.dll (Java Virtual Machine) to be available in the PC's path. 4.2.3 How to run Adaptor To run the Adaptor to view the movements extracted by Extractor, to verify them, to modify them if necessary and to pass them to the XML queue follow these steps. " Start Adaptor from the Desktop (the Messages tab will be displayed). View the movements and make modifications to the data or status if necessary (remembering that movements that have already been submitted successfully cannot be modified). When you are satisfied, click on the Submit tab. " Ensure that the Source is set to iBroker (if necessary) and click on the appropriate radio button to select how you would like to process the movements. Initially it would be best to select - Process 'Ready to Submit' one by one. Once this is selected the 'Next' button will be activated. Alternatively if you click on Process all 'Ready to Submit' messages you will need to click on the Submit button to start the process. To process just one specific message, click on the message to be processed in the while display panel, click on the 'Process one selected message' radio button and then click on Submit. " The status of each message will be displayed in the yellow 'Result' area at the bottom of the screen. The first message processed each run will take a longer time because libraries (programs) are loaded during the first message processing. If a message fails it is best to note the reason and the message details. Once the run is completed, you can go back into the Messages screen, change the Status field to Blocked, Rejected or Bypassed depending upon the error message generated and then click on the Search button. Then click on the failed movement and 'Modify' it, if possible. You must then save the 'modified' message. Once this is complete (including setting the status back to Ready to Submit), you are able to go back into the Submit tab and try again. " To exit, click on the X in the top right hand corner of the FM-Main window. 4.3 Release To run the Release process, start m2Controller from the shortcut on the desktop. This shortcut is labelled M2FController and will have been set up for you by M2. 4.3.1 Overview of Release The Release process enables the user to release the movements in the XML queue to the Submission process which transfers the movements to the M2 Service. This process requires minimal manual input at present but will be automated in time. 4.3.2 How to set up Release The Release process does not require any configuration to run. 4.3.3 How to run Release To run the Release process to take the movements in the XML queue and to Release them to the Submission process follow these steps. " Start m2Controller from the Desktop. It will bring up a screen that initially displays the Clients in M2CRM. At the top of the screen are 3 tabs labelled Client Information, Holdings and Movements. At the right of the screen are 3 tabs labelled Client, Interfaces and Sys Admin. If you log on using a Userid that does not have the correct privileges then one or more of these tabs will be hidden. " Click on the tab at the right titled Interfaces. This will display on the left hand side the various categories and numbers of messages awaiting Release. Messages may have a status of Error, Waiting Release or Released. At a later date there will be other message types but for now only M001 messages will be displayed. If movements have been passed through the Adaptor successfully, they will be displayed at Awaiting Release. " Click on the Awaiting Release words and all movements that are in the XML queue ready for release will be displayed in the panel on the right. At the bottom of the screen are 7 buttons. Buttons that are greyed are not available for this selection. Initially, Select All, Deselect All and Search will be enabled. " To Release all the waiting messages, click on the Select All button and all messages will be highlighted. At this point the Release, Reject and Status History buttons will be enabled. Click on Release to Release all messages to the Submission process. This process will allow you to release these movements to the Submission process, which is operating in background and will send the movements to the M2 process. " You are able to view the history of all movements via the Status History button and you are able to restrict the movements in the display area (and subsequently those released) by using the Search button and changing the selection criteria. For instance, you are able to restrict the release process to only those messages for a particular day using the Search button. The Search button is activated by clicking on it and de-activated the same way. " After clicking on the Release button you are asked to confirm the Release on the selected movements. Click on OK to proceed or cancel to go back. After the messages are released they will move from the Awaiting Release queue to the Released queue. If any messages fail they will appear in the Error queue for further processing. Error messages can be Rejected (removed permanently) or can be edited and re-released. " To exit m2Controller after the messages have been released, click on the Logout button at the bottom of the screen. 4.4 Submission The program to run this is called runMvmtSubmission.bat and is located within the C:\Program Files\m2Controller\adaptorUtils directory. A shortcut to this program called M2FSubmission will be created for the client by M2 operations staff. This will be called M2Submission. 4.4.1 Overview of Submission The Submission process enables the user to submit the movements that have been released from the XML queue to the M2 Service. This process requires only to be started at present but will be automated in time. 4.4.2 How to set up Submission There are no requirements to set up the Submission process. 4.4.3 How to run Submission A shortcut has been created on the Desktop called M2FSubmission. Clicking on this icon will start the Submission process which then sits in background waiting for movements to be released. No other input is required by the user, except to re-start the Submission process each time the PC is logged in to. A black screen with writing will appear in the middle of your screen. This can be minimised by clicking on the - sign at the top right of the screen. This screen will however, display progress and error messages so it is advisable to check the screen before closing. To close, click on the X sign at the top right of the screen. After the Submission process is completed, log back into m2Controller to check the status of the Submitted messages. To check the status of the messages, follow these steps. " Start m2Controller from the Desktop icon. Click on the Interfaces tab at the right. If the messages were successfully submitted then the three queues will be empty. On the other hand, if messages failed they will be displayed in the Error queue. " Click on the word Error to display the messages in this queue. The list of messages in error will be displayed in the right panel. For each message there will be a + sign to the left of the message. To display the reason for the failure, click on the + and a line with the error message is displayed. Often the failure will be as a result of processing the same message twice. " If you wish to re-release a message, click on the message and then click on the Release button. You might wish to do this if the reason for the error was that the M2 Server was not available when the message was submitted. Alternatively, if you wish to re-release all messages then click on the Select All button and then the Release button. " You are able to display the history for each message by clicking on the message and then clicking on Status History. " If the Rejection of the Movement was valid (for instance duplicates through running the process twice), click on the message that is in error (or if all messages Click on Select All) and then click on Reject. You will be asked to enter a Reason for the Rejection. If you have run the messages twice, enter some text to this effect. The number of messages will be decremented after the Rejection is actioned. " To logout, after you have re-processed all error messages, and re-released those that are in the Awaiting Release queue, click on the logout button at the bottom of the screen. 4.5 Update This process is a Service running on the M2 Central Server. It is always running in listen mode awaiting any movements sent to it by the Submission step on the client site. 4.5.1 Overview of Update This process receives movements sent to it by the Submission process at the client site. It writes the movements into the specified end-users Account(s) updating the Holdings to match the positions held within the in-house systems. In general, as the Interface will only be run once a day and either within the End-of-day process or first thing in the morning, the updates to the end-users Portfolio will only be current after this event. 4.5.2 How to set up Update No Client input is required into this process. The service is controlled by M2 and is always running. 4.5.3 How to run Update No Client input is required into this process. The service is controlled by M2 and is always running.